Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-5307

Add utility to help with NotSerializableException debugging

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.3.0
    • Spark Core
    • None

    Description

      Scala closures can easily capture objects unintentionally, especially with implicit arguments. I think we can do more than just relying on the users being smart about using sun.io.serialization.extendedDebugInfo to find more debug information.

      Attachments

        Issue Links

          is duplicated by

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-3694 Allow printing object graph of tasks/RDD's with a debug flag

          • Major - Major loss of function.
          • Resolved
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-3694 Allow printing object graph of tasks/RDD's with a debug flag

          • Major - Major loss of function.
          • Resolved
          links to

          Pull request #4093 [Github] Pull Request #4093 (rxin)

          Pull request #4098 [Github] Pull Request #4098 (rxin)

          Pull request #4297 [Github] Pull Request #4297 (rxin)

          Activity

            People

              rxin Reynold Xin
              rxin Reynold Xin
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: